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Overview of the Contents 



I magine yourself the designer of a large-scale arts assessment. You are the person in your district or state 
responsible for coordinating the assessment of student learning in the visual and performing arts. The task 
will require you to make several important conceptual, technical, and administrative decisions. 

This handbook is a guide to help with the decisions you face in the design of an arts assessment. It poses 
important questions for you to consider and helps you answer them. It also offers informational support and 
useful activities that will generate various strategies and assessment design options for you to consider. The 
guide is intended to be both a practical tool and a reference. You may want to browse through the entire book- 
let first to get a full picture of what it covers, then go back to various chapters when you are ready to work 
through them. Several concepts are discussed briefly, but specific references to other sources that contain fur- 
ther descriptions of them are included. 

In this guide, you will find 

1. A brief background on the standards and assessment movement 

2. A basic three-phase plan to orchestrate your district or state arts assessment 

3. A set of flexible steps to lead you through planning, developing, and implementing your program 
design 

4. Activities that will help you address key issues and make administrative decisions to carry out your 
assessment 

5. Concrete examples drawn from existing state arts assessment efforts, and the National Assessment 
of Educational Progress (NAEP) Arts Assessment 

The first part of this guide reviews the thinking behind large-scale assessment. The topics briefly addressed 
include the history, theory, and values that surround the assessment movement. The summary will serve to 
remind you and the committees who work with you of the original intent of standards-based reform. Other 
sections in the opening highlight the contributions of the 1997 NAEP Arts initiative and the rationale for 
launching an arts assessment program. 

The second part of the guide focuses on the practice of large-scale arts assessment. It emphasizes the appli- 
cation of concepts and suggests a sequence of phases, steps, and activities in an assessment design process. 
Each phase highlights key responsibilities and challenges. Specific examples, useful ideas, and common ques- 
tions are included to support your efforts in each stage. The final chapter discusses the benefits of assessment 
to arts education. 

Whether or not you have expertise in the fine arts or assessment, you should find this guide a useful resource. 
It will help you examine your assessment activity and shape a program in reference to your district or state 
arts standards. Much thoughtful, significant work has already taken place in the field of large-scale assessment 
across many subjects, including the arts. One intent of this book is to draw upon and share the insights of 
these efforts. The other is to guide the design of an arts assessment compelling enough to inform and strength- 
en arts education for all students. 
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Part One 



Background for Thoughtful Arts Education Assessment 



Assessment has both common sense and 
technical meanings. We assess a situation as 
the basis for deciding a future course of 
action. Assessing how well students are 
learning in the arts is an essential and daily 
part of teaching. But assessment has taken on 
new significance - and new technical 
demands - with the rise of the standards- 
based movement for school improvement. 
Now we need to know how a school, school 
district and an entire state is doing to help all 
students meet established expectations for 
learning, including learning in the arts. So we 
need "large scale" assessments of students 
that often have major consequences in terms 
of promotion and graduation for students or 
financial incentives for schools and teachers: 
these are "high stakes" assessments. Where 
did this movement come from? What are its 
benefits? Flow do we measure the unique 
learning that occurs in the arts with large 
groups of students? What can we learn from 
the experience of the federal government in 
developing and conducting the 1997 
National Assessment of Educational Progress 
in the arts? The following chapters explore 
these questions, offer answers, and suggest 
resources for to you consult to guide your 
thinking and action. 
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Chapter 3 



Why Take on the Challenge of Assessment? 



S tate and district arts education leaders around the country have been 
working to establish effective arts education assessments in order to get 
the arts "on the table" of taught curriculum. Statewide arts assessment 
efforts are in constant flux and often suffer from inadequate public under- 
standing about the value of arts education and a low priority compared to 
other subjects. 

While this can be discouraging, it should not imply that arts educators aban- 
don efforts for statewide arts assessment. As of this writing, 1 7 states have 
initiatives in place that support statewide arts assessment and still others are 
pending. The very process of working toward a statewide arts assessment 
yields value for state and local arts educators. By participating in the design 
of assessment frameworks and tasks, arts teachers and curriculum specialists 
increase their knowledge of effective arts curriculum, instruction, and class- 
room-based assessment. Even in states where arts assessment has moved to 
the back burner, local arts educators have benefited from the development 
process. 

Linking Large-Scale Assessment with Classroom Assessment Purposes 

With the growing interest in performance assessment, statewide assessment 
efforts in other subjects provide interesting learning tools for prospective arts 
assessors. Vermont, for example, is well known for its pioneering attempts 
to link large-scale assessments with classroom efforts through its portfolio 
performance assessments of writing and math. Teachers from across the 
state scored portfolios using rubrics and exemplars and the professional 
development of the teachers was an important aspect of the system. Even 
though the system did not meet its goals in terms of large-scale accounta- 
bility, local administrators and teachers saw positive impact of the process 
on instruction. (Eliot Asp, "The Relationship Between Large-Scale and 
Classroom Assessment," Assessing Student Learning, ASCD, 1998, pp.31- 
32). 

Kentucky also attempted to incorporate large-scale and classroom assess- 
ment into the same system. While such attempts are appealing, Asp cau- 
tions that by attempting to serve a variety of purposes with one assessment 
system, "the system may not be served very well. The solution or middle 
ground is not yet in sight." (Asp, p. 35) 

Much is still to be learned about performance assessment in general, and 
about arts performance assessment in particular. As increasing numbers of 
arts educators take on the challenge of arts assessment, it will be important 
to share the work widely in order to collectively advance the field. 



At the individual state 
level , the ebb and flow 
of education reform ini- 
tiatives have alternately 
fostered and under- 
mined arts assessment. 
States tend to develop 
assessment of core cur- 
riculum in a classical 
order that most educa- 
tors can recite by rote: 
English/Language Arts, 
Math, science, social 
studies, (then) the other 
areas - including the 
arts - as time and fund- 
ing permit. Few if any 
states have been able to 
sustain their education 
reform efforts long 
enough to 'get around” 
to the arts. 

Scott Shuler and Selena 
Connealy, " The 
Evolution of State Arts 
Assessment: From 
Sisyphus to Stone 
Soup”, Arts Education 
Policy Revie w, Vo I. 1 1, 
No. 1 , Sept/Oct 1998., 
p. 14 . 
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Linking Arts Efforts to Large-Scale Assessments in Other Subjects 



One can travel anywhere in the country today and ask local teachers for the 
current acronym of high stakes tests in their state: FCAT in Florida (ef-cat), 
WASL in Washington State (wazzul) and so forth. In states where scores on 
these tests are tied to high stakes accountability measures, teachers describe 
the very real impact of these tests on the classroom such as endless practice 
tests Cdrill-and-kiir) and mandated increased instructional time spent only 
on tested subjects. 

It is not surprising that arts educators worry that this kind of test-prep frenzy 
may lead to decreased instructional time in the arts. It would be wise for 
arts educators to become informed about high stakes tests in their districts 
and states in order to examine sensible linkages between arts learning and 
other tested content: 

1 . Take a look at the test preparation materials . What kinds of ques- 
tions are being asked? What kind of visual symbols and images are 
used? What are students being asked to think about and how? 

2. Stay current Anyone who has not recently reviewed tests of writ- 
ing or math may be surprised by the changes. Students are increas- 
ingly being asked to "show your thinking" in math problem-solving 
tasks. More and more, student writing is being scored not solely by 
grammar and spelling, but according to such traits as "voice" and 
the student's ability to "match voice to intended purpose and audi- 
ence." (Washington Assessment of Student Learning in Writing) 
Similarities between this approach to writing and approaches to 
composition in the arts are obvious. 

3. Gather arts educators to analyze state assessment materials 
together . Explore sensible links that can be made with authentic 
learning in the arts. 

4. Encourage conversations among arts teachers and classroom col- 
leagues - grade level elementary school meetings or cross-depart- 
ment meetings at the secondary level - where teachers can begin 
conversations about important linkage and learning across content 
boundaries. 

While the controversy rages about whether or not the arts should be instru- 
mental to learning in other subjects, the fact remains that arts specialists and 
classroom teachers share the same students. Arts teachers are part of the 
fabric of the total school and, as such, face the same challenges posed by 
high-stakes testing. By familiarizing themselves with expectations for their 
children on high-stakes tests in other subjects, arts specialists can discover 
sensible and legitimate connections with important arts learning. 
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Chapter 4 



Features of Effective Large-Scale Assessment Design 



W hen considering large-scale assessment of arts education, arts 
administrators and program coordinators will be able to refer to 
several of the assessment initiatives mentioned in this handbook- 
initiatives that provide important lessons for envisioning your own assess- 
ment needs. Here are four important features to consider; 



1. Take time. The large-scale assessment programs that were studied 
took three to five years from inception to final implementation, with 
a full year devoted just to planning. All demonstrated that multi- 
year effort is required to fully develop several phases of activity. 
Further, when statewide assessment efforts faced problems and crit- 
icism, the reason often cited was too little time for building teacher 
and public support for the program. 

2. Understand the distinct phases of activity along with the assis- 
tance and funding required for each. As successful projects have 
shown, activities such as writing test items and tasks involve plan- 
ning, design, development, review, and revision, and field-testing. 
Adequate funding is needed to carry out each process, which may 
also require the help of specialists versed in areas such as test for- 
mat, measurement, professional scoring, and reporting approaches. 
The matter of funding is critical particularly in a large-scale arts 
assessment that requires the purchase of specialized consumable 
supplies, visual art reproductions, and audio and video equipment 
and technology. 

3. Apply collaborative strategies from the outset. More than any 
other reform undertaking, formal assessment can be a volatile issue, 
especially in the arts. For this reason and others, the assessment 
agenda should be mutually determined and controlled by those 
with program responsibility and a vital interest in arts education, 
including parents, students, administrators, project sponsors, artists, 
and arts teachers. 

Repeatedly, effective programs point out the value of including arts 
practitioners throughout each phase. One promising model 
required that teachers be actively involved in developing, scoring, 
and interpreting the assessments. Only in this way will those who 
must implement the system become fully committed to it. As one 
theory and practice study found, when teachers participate in mak- 
ing sense of student responses and the evidence of learning, their 
understanding is deeper and more meaningful than if they relied on 
another person's interpretation. 
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As compared to meas- 
urement , assessment is 
inevitably involved with 
questions of what is of 
value , rather than sim- 
ple correctness. 
Questions of value 
require entry and discus- 
sion. In this light , assess- 
ment is not a matter for 
ours/c/e experts to 
design; rather, it is an 
episode in which stu- 
dents and teachers 
might learn , through 
reflection and debate , 
about the standards of 
good work and the rules 
of evidence. 

Wolf, Bixby , Glenn , & 
Gardner 



4 . Link assessment to the same standards across district and state 
schools. Although this seems quite obvious, reviewers of state 
assessment documents report that alignment between assessment 
and standards is often weak. To better ensure the linkage, strong 
assessment programs maintain internal control over the develop- 
ment of their performance tests in contrast to relying completely on 
commercially-prepared versions. This also ensures that thinking 
skills in the arts disciplines are emphasized and that students are 
required to apply what they know to solve complex arts-based 
problems. Successful assessments also forge strong links to curricu- 
lum and professional development. 
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Part Two 



Assessment Design in Action 



Planning, development and implementation 
constitute the key phases in an assessment 
design. The following pages make clear the 
stages and steps that you will have to con- 
sider carefully to ensure a successful large- 
scale assessment of learning in the arts. Each 
chapter reflects background research of 
leading state and national large-scale efforts 
in the area of assessment and the arts. As 
you read through the design phases, consid- 
er each step and activity as an of a flexible 
and reflective process whose purpose is to 
be meaningful and useful to you and your 
diverse stakeholders— students, teachers, 
parents, and the district and state board of 
education initiatives already in place. 
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Chapter 5 

Plan and Clarify 



K ey steps in the planning phase include: 

1. Identify members of a planning and design committee. 

2. Establish the purpose(s) of your arts assessment program. 

3. Develop guidelines for the design and implementation of the 
assessment. 

4. Create a timeline for accomplishing major milestones. 

5. Examine and learn from existing large-scale assessment and arts 
efforts. 

Step 1: 

Identify members of a planning and design committee. Think about invit- 
ing individuals with a vital interest in arts education who can serve in an 
advisory or steering capacity throughout program development and imple- 
mentation. Select members to represent constituencies important to your sit- 
uation, such as those suggested. 



Activity: Identify planning committee members 



Arts teachers for dance, music, theatre and visual arts, also from various size 
districts and statewide regions, grade levels, and diverse backgrounds 

State Education Department staff and consultants such as those for testing, 
measurement, and arts curriculum 

School district administrators such as superintendents, principals, curriculum 
and instructional supervisors 

Professional development regional facilitators 

Parents 



Higher Education faculty in the arts and teacher education 

Community representatives such as members of relevant professional 
organizations, older art students, practicing artists, political 
arts advocates, and other program stakeholders 
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Step 2: 



Establish the purpose of your arts assessment program. With your planning 
group, determine your assessment goals. You may want to prioritize them if 
you have more than one. Remember, different kinds of evidence must be 
gathered to suit different purposes. Here are examples of district and 
statewide goals for conducting a large-scale arts assessment. 



For a complete list of 
the 17 states with arts 
assessments and the 
goals and essential char- 
acteristics of their pro- 
grams , see State-of-the- 
Art in Large-Scale Fine 
Arts Assessments , a 
report by Westat, 2001 
for the Maryland State 
Department of 
Education , listed in the 
references section. Also , 
CCS SO publishes , an 
Annual Survey of State 
Student Assessment 
Programs , which 
includes information on 
states that are develop- 
ing or have developed 
arts assessments . 







Population/level Served 


Sample Goal 


Students 


To determine student achievement in 
meeting arts standards; for graduation requirement 


Teachers 


To improve classroom arts instruction 


Schools 


For school-wide accountability and to 
improve arts curriculum and programs 


District/State 


To establish district/state profiles of student 
achievement in one or more of the arts disciplines 



A review of the 17 states currently involved in large-scale arts assessment 
reveals two common purposes: to establish a state profile of student 
achievement in dance, music, theatre, and the visual arts; and to improve 
instruction and learning in these areas. One of these states uses its assess- 
ment results for school accountability and one as a graduation requirement. 

Step 3: 

Develop guidelines for the design and implementation of the assessment. 

Once your assessment goals are established, the planning committee can 
begin to determine program guidelines by addressing the important ques- 
tions below, and by identifying others that may be relevant to your situation. 
The decisions you make about these matters will provide a common refer- 
ence point as you move forward with each phase of development. 
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Activity: Discuss these questions to determine your 
assessment program guidelines and characteristics. | 


Topic/Question 


Example 


Program Features 




Which arts disciplines will be assessed? 


Dance, music, theatre and/or visual arts 


What grade levels will be assessed? 


Grades 4, 8, and 1 2 such as NAEP, or others 


Will the arts assessment be a stand-alone 
test or will fine arts questions be included 
on other subject area achievement tests? 


A separate arts test for grade 10; fine arts questions 
included on the Social Studies achievement test 
for grades 4 and 7 * 


What type of test will it be? i.e, paper/pencil 
(written), performance or portfolio? 


A combination of written and performance 
tasks with a portfolio component.* 


Will the test be mandatory or voluntary 
for schools and districts? 


Schools have the option of administering the 
voluntary fine arts test by completing an 
enrollment form provided to principals by the 
state's division of assessment * 


Test Development 




Who will develop the fine arts test? 


Arts teachers across grade levels will be recruited, 
trained and hired by the state to write test items 
and pilot them in classrooms. Additional experts 
will be identified to review items for content 
concerns and bias. Specialists in test writing will 
assist teachers. 


Test Administration 




When will the assessment be administered? 


The arts test will be administered each spring 
during a five-week "test window." Districts may 
schedule the test anytime during that period.* 


How will the arts test be administered? 


District schools will administer the (voluntary or 
mandated) test during the state's achievement 
testing period. Testing materials are sent two 
weeks prior to this timeframe* 


What professional development will be 
available to districts prior to 
administering the arts assessment? 


Procedures for administering the arts test will be 
reviewed as part of regional meetings held to 
address questions pertaining to subject area 
achievement tests.* 


Are special education students included 
in the arts test? 


Special education students are included. Individual 
Education Plan (IEP) team members decide whether 
any test accommodations are necessary. 


Are students with Limited English 
Proficiency (LEP) included? 


LEP students are included with accommodations 
either specified in a student IEP or frequently 
used to test the student. 


Scoring and Reporting 




Who will score the fine arts tests? 


State arts teachers assisted by assessment specialists 
will score the tests. The state will hire and train 
teachers over a three-week period in June to 
conduct the scoring at regional scoring sites.* 


How will scores be reported? 


Results may be reported by total (composite score), 
by arts discipline (percent scaled right), by level 
of achievement, or by state art standards. 


Who will receive test results? 


Individual and group results will be provided at 
the district, school and student levels.* 


Resources 




What resources will be available to assist 
teachers in helping students perform well? 


Visual and performing arts standards; curriculum 
frameworks; regional assessment facilitators; 
fine arts released test items. 



Examples followed by an 
asterisk (*) are drawn from 
actual assessment programs 
to show how others 
addressed the question . 
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Step 4: 



Create a timeline for accomplishing major milestones. As mentioned pre- 
viously, most effective large-scale assessments provide for multi-year effort. 
With a broad view of the distinct phases of activity required, you should set 
a timeframe for completing important responsibilities at each stage. 
Remember, too, that it will take considerable time to build arts teacher 
capacities to participate in the assessment, as well as to garner community 
and public support for the program. 



Activity: Establish a timeline for reaching target milestones. 
To be realistic about the time required, consider your staff 
experience levels and available resources. 

(The guide's 3 phases & steps will help identify milestones.) 


(Number of Years Optional) 


Selected Example Milestones 


Planning, Year 1 


n Select planning/advisory team 
D Establish program goals 
O Determine all assessment design features 
including professional development needs 

n Study effective district/state arts testing 
models 

Develop framework and test/item 
specifications (blueprint) for the arts 
assessment 


Development, Year 2 


□ Engage arts teacher participation in 
all aspects of test development 

n Establish regional sites for training test 
writers and scorers 

□ Generate a "pool" or bank of items/tasks; 

conduct pilot tests and refine items 

□ Provide regular updates to districts 

about the assessment 


Initial Implementation, Year 3 


O Coordinate the field test by selecting 
schools and producing test booklets 
and materials 
D Conduct the field test 
□ Analyze field test data to select and refine 
items and scoring guides 


Full Implementation, Year 4 


O Announce the voluntary or required 

assessment system-wide with all details for 
enrollment and participation 

□ Administer the arts assessment during 
year-end state achievement tests 
O Analyze data and report findings 
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Step 5: 



Examine and learn from existing large-scale assessment and arts efforts. 

Over the past ten years, there have been significant accomplishments in arts 
education assessment. National organizations and state agencies have 
experimented in the fields of large-scale assessment and the arts to develop 
well-tested program models and to provide practical guidance. It is safe to 
say that most inquiries you have, challenges you face, or examples you need 
in launching your district or state program, are specifically addressed in one 
or more of the detailed sources listed below. Having this material available 
at the outset will save you and your committees from "reinventing the 
wheel." 



Activity: Gather information sources that help clarify 
large-scale assessment concepts and which illustrate 
arts assessment program ideas. 



Top Choices to Get Started 

O Arts Education Assessment Framework , a publication prepared by CCSSO for the 
National Assessment Governing Board (NAGB), 1994. Provides a blueprint that 
guides the development of an arts assessment. Organizes the assessment around the 
artistic processes common to all four arts areas-creating, performing, and responding. 

O Arts Education Assessment and Exercise Specifications , a publication prepared by 

the College Board for NAGB, 1994. Provides an overall description of the construction, 
review and scoring of test items in drama, music, theatre, and visual arts. Also includes 
guidelines for selecting stimulus material in each arts area. 

□ The NAEP 1997 Arts Report Card, Eighth-Grade Findings from the National Assessment 
of Educational Progress , 1998. U.S. Department of Education, Office of Educational 
Research and Improvement. This report includes sample test items and tasks in all 
four arts disciplines with scoring criteria. 

□ State-of-the-Art in Large-Scale Fine Arts Assessment, a report by Westat for the 
Maryland State Department of Education, 2001 . The most current listing of all states with 
mandated or voluntary visual and performing arts assessments. Includes the essential 
characteristics of each program including item format, reporting strategies, lessons 
learned, and contact information. 

D Critical Issues in Large-Scale Assessment, a publication of the Council of Chief State 
School Officers (CCSSO), 2001 . A Resource Guide to provide practical guidance and 
support for the sound design, development, and implementation of large-scale 
assessment. 

□ Handbook for Professional Development in Assessment Literacy, a publication of the 
Council of Chief State School Officers (CCSSO), 2001 . A resource to states and districts 
as they develop and implement assessment systems aligned to standards for purposes 

of improving student learning through accountability and school improvement. Includes 
a complete glossary of relevant terms to the assessment field. Also included are 
presentation materials designed to increase the assessment literacy of state and district 
staff, teachers, parents and policymakers. 

On-line Sources 

□ The Arts Education Partnership: http://www.aep-arts.org 

O NAEP web site: http://nces.ed.gov/nationsreportcard/arts/ 

□ CCSSO - SCASS Arts Assessment web site: http://www.ccsso.org/scass/p_arts/index.html 
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Chapter 6 

Develop and Generate 



K ey steps in the development phase include: 

1. Organize test development teams experienced in the arts disci- 
plines to be assessed. 

2. Create a framework to capture the range of content in your arts 
standards. 

3. Write test items and performance tasks with scoring guides based 
on your framework specifications. 

4. Review assessment items to identify areas that require change or 
revision. 

5. Pilot test the items and tasks in school district classrooms. 

Step 1 : 

Organize test development teams experienced in the arts disciplines to be 

assessed. Almost all states recruit, hire, and train classroom arts teachers for 
their writing teams. They also hire assessment specialists and include other 
arts discipline experts to assist in the development process. There are sever- 
al nonprofit education agencies in the assessment field that have consultants 
that can help facilitate all aspects of test development and scoring, as well 
as train teachers in these areas. On the next page is a sample memo used by 
one state to recruit teachers for committee membership. Note the proce- 
dures followed and the qualifications required for teacher participation. 



Activity: Recruit arts teachers statewide to write test questions and exercis- 
es. Hire additional assessment specialists from external agencies as need- 
ed. (The sample memo on the next page will help you identify the aspects 
of recruiting and preparing teachers to participate on writing teams.) 
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STATE DEPARTMENT OF EDUCATION 
2002-2003 ARTS ASSESSMENT DEVELOPMENT 
COMMITTEE MEMBER RECRUITMENT 

/s/\E/V\OR/XM d u m 



To: Superintendents, Principals and Arts Content Specialists 

From: Director, Division of Curriculum and Assessment 

Date: May 1 9, 2002 

The Department of Education is seeking State classroom arts Teachers in the 
areas of dance, music, theatre and visual arts for a limited number of memberships 
on the assessment development committee for the State's Fine Arts Assessment 
Program. Committee members work with the Department of Education's assessment 
coordinator, contractor and other specialists to develop, review, and refine visual 
and performing arts test questions. 

Committee members must have content experience in one arts discipline, 
understanding of and direct experience with the State's Arts Curriculum Frameworks 
and Standards, and experience at the grade levels being tested. We are looking 
specifically for teachers in the following grade levels and art subject areas: 



Grade 4/5 


Dance 


2-3 teachers 


Grade 4/5 


Music 


4-5 teachers 


Grade 4/5 


Theatre 


2-3 teachers 


Grade 4/5 


Visual Arts 


4-5 teachers 


Grade 9/10 


Dance 


2-3 teachers 


Grade 9/10 


Music 


4-5 teachers 


Grade 9/10 


Theatre 


2-3 teachers 


Grade 9/10 


Visual Arts 


4-5 teachers 



Committee members are required to attend a three-day summer meeting and at 
least two meetings during the school year. Other meetings may be called as necessary. 

The Department will reimburse districts for substitute teacher expenses for meetings 
during the school year. Meals, mileage, and lodging (when necessary) are provided 
by the Department of Education. Committee members will receive an honorarium 
for attending summer meetings. 

Please consider eligible staff and encourage such persons to complete and submit 
the enclosed application. Completed applications must be received by 5:00 p.m., 

June 1 6, 2002. 
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Step 2: 



Create a framework to capture the range of content in your arts standards. 

The framework is a critical feature of assessment design. Its purpose is to 
specify the knowledge and skills to be assessed based on your arts stan- 
dards, or in other words, to determine the content of assessment questions 
and tasks. Think of the framework as the blueprint that will guide test item 
development. You can use it to write and distribute questions among con- 
tent strands that reflects the relative importance and value you give to each 
strand within an arts discipline. You may want to create a separate frame- 
work for each arts discipline to address its special characteristics. The NAEP 
theatre assessment framework is shown below as an example. You want to 
create your framework(s) in reference to your district/state arts standards or 
learning outcomes and curriculum. 



Activity: Create a framework for each arts discipline to be assessed to 
determine the content of test items and performance tasks. 

EXAMPLE 



NAEP THEATRE ASSESSMENT FRAMEWORK 



ARTS PROCESSES FOR THEATRE 

CREATING/PERFORMING 

When creating and performing in theatre, students: 

□ develop scripts and scenarios; 

□ develop characters through an acting process; 

□ make design and technical choices to communicate locale and mood for 

dramatic material for theatre, film and television; and 

□ direct by interpreting dramatic texts and organizing time and people in planning and 

rehearsing improvised and scripted scenes. 

RESPONDING 

When perceiving, analyzing, interpreting, critiquing, and judging works in theatre, students: 

□ describe and analyze artistic choices in their own work and construct meaning; 

□ describe and compare elements, styles, genre, media, and dramatic literature; 

□ place work in context {personal, social, historical, and cultural); and 

□ evaluate performances as audience and critic. 



BASED ON SPECIFIC CONTENT FROM THEATRE 



Applying KNOWLEDGE of: 

• Different contexts for theatre 

- personal, 

- social, 

- cultural, and 

- historical 

• Aesthetics 

• Form and structure of theatre 

• Theatre processes 



Applying cognitive, affective and 
motor SKILLS in: 

• Perceiving 

• Intellectual/reflective behavior 

• Expressive behavior through theatre 

• Technical mastery of theatre content 
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Step 3: 

Write test items and performance tasks with scoring guides based on your 
framework specifications. Besides the content emphasis of the questions, 
the development committee also will consider the format of test items 
before writing. There are various types of questions from which to choose. 
The NAEP arts exercise guide suggests that the acceptable formats for arts 
assessment include: 

❖ constructed-response exercise formats 

♦ production/performance tasks 

♦ open-ended questions requiring short written responses 

♦ open-ended questions requiring extended written responses 

❖ selected-response (multiple-choice, matching, etc.) items 

Test developers should consider allocating a greater percentage of testing 
time to the item formats that best elicit the range of student abilities valued 
by the arts content and performance standards and identified in your frame- 
work. For example, NAEP framework specifications recommended that the 
1 997 arts assessment be designed so that students spend 80-90% of the time 
working on constructed-response exercises and 10-20% of the time on mul- 
tiple-choice questions. 

Your development committee will find it useful to create a grid, like the one 
below, which specifies the number of questions by type and arts area. 
Writing test questions begins once the framework specifications are in 
place. 



Grade 8 

Number of Questions by Type 





Dance 


Music 


Theatre 


Visual 

Arts 


creating/performing 

excercises 










short 

constructed-response 










extended 

constructed-response 










selected-response 

(multiple choice, matching, ctcJ 










TOTALS 











See The NAEP 1997 Arts Report Card for examples of how various question formats are used and com- 
bined in a large-scale arts assessment. 
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Developing Scoring Guides for Test Items and Tasks. A scoring guide has 
three components, each of which contributes to its usefulness: 

❖ Criteria that serve as the basis for judging the student response; 

❖ Achievement/performance levels for the criteria: a rating scale that is 

either numerical such as a point scale from 1-4; or qualitative with 
levels such as basic, proficient and advanced; and 

❖ Descriptors that clarify the meaning of the criteria at each achieve- 

ment level (and for each grade level). 

Developing scoring guides is a multi-stage process. As questions are devel- 
oped, test writers should articulate performance criteria and draft an initial 
version of the scoring guides. Arts content and measurement specialists 
should help review the scoring guides to ensure that they include criteria 
consistent with the wording of test questions; also, that they are concise, 
explicit, and clear and reflect the assessment framework. Next, the draft 
guides are used to score student responses in pilot and field-testing. The 
results of these trials are used to further refine the guides. 

Pages 28-30 provide a visual arts "performance event" developed by the 
New York State Education Department and provided here with permission. 
Note how the example illustrates how a performance task is constructed and 
the use of a scoring guide with its three components. 

Activity: Prepare your writing committee to develop arts test items and scor- 
ing guides by examining the example that follows for a visual arts perform- 
ance task. Discuss the tasks' content, format, timing, directions to the stu- 
dent and scoring criteria. (See the Appendix for sample items/tasks and scor- 
ing guides from a performance exercise in dance.) 
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Performance Event Item 

Used with permission of the New York State Education Department 



Note: This image has been reduced. 




The City Rises by Umberto Boccioni, 1910 



Directions: Carefully study the drawing The City Rises, by Umberto Boccioni, 
and then read the information below. You have five minutes to read Part 1 and 
study the image. 

Part 1: Analysis of the Drawing 

Look at the drawing The City Rises ; by Umberto Boccioni. In this work the artist uses line, value 
contrast, and composition in certain ways to create a sense of dynamic action and tension. 

• Line 

Boccioni uses lots of short, quick diagonal lines to build up his images and create a sense of energy 
and movement in the artwork. What other kinds of lines or marks could you use to create a sense of 
dynamic action and tension? 



Visual Arts 
Standards 1 &3 

Performance 

Indicators 1(a), (b), (c), (d), 
3(a), (b) 



Continue on the next page 
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• Contrast 

Boccioni also uses contrast to help draw your attention to the central action in the drawing. The massive 
horse-like image in the foreground stands out against the lightness of the ground underneath and 
to the left of him, drawing our attention to the curve of his powerful head. What other ways could 
you use contrast to draw attention to the main theme of a work? 

• Composition 

Certain choices about composition also affect the sense of dynamic action and tension in this work. 
Notice how much space the main horse takes up in the entire composition, and how the diagonal 
movement of his body and other images in the drawing lead your eye across the picture plane. What 
other compositional arrangements could be used to create a sense of action and tension? 

Part 2: Sketches for a Composition 

Directions: Now you are going to draw two sketches of your own hand that create a sense of dynamic 
action and tension. 

REMEMBER: A sketch is quick and somewhat rough, not a finished drawing. The purpose of these 
sketches is to help you develop ideas for a dynamic composition that draws attention to your hand as 
the main image in your drawing. 

• Sketch two ideas in preparation for your final composition. Each of your 
sketches should feature your hand as the central or most important image. 

• Each sketch should explore a different dynamic composition. You may try 
different poses of your hand and you may work horizontally or vertically. 

• Consider all the space in the rectangle as part of your composition. 

• You may add other shapes or images to the background to enhance the 
dynamic action of your composition. 

You have 15 minutes to complete these sketches. 

If you complete the two sketches before the 15 minutes are up, you may move on to Part 3. 

Note: Students are provided with two rectangles 3 3/4"x T1 in which to sketch. 
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Part 3: Final Drawing 

Directions: Select your most dynamic composition from Part 2, and develop it into a final drawing. 

The final drawing should be completed in a more finished manner, using line in a way that enhances 
the sense of tension and dynamic action, and contrast to draw attention to the central focus in your 
drawing. 

Do the final drawing in the rectangle provided. 

Note: You may look back at Parts I and 2 to help you as you complete this part of the task. 

It is all right to alter your final drawing somewhat from your sketch in order to improve upon it in 
any way. This final drawing will be evaluated on: 

• the use of COMPOSITION to create a sense of dynamic action and tension; 

• the use of LINE to enhance the sense of dynamic action and tension; and 

• the use of CONTRAST to draw attention to the central focus in your drawing. 

You will have (25) minutes to do Part 3. Do not continue on to Part 4 when you finish Part 3. 

Scoring Guide for Performance Event Final Drawing 

Criteria for final drawing: 

• Skillful use of value contrast 

• Careful consideration of the whole compositional space 

• An effective use of line 

• Hand must be the main focus. 



Level 


Description 


4 


The final drawing is a visually dynamic composition in which all the criteria are 
met in a highly successful manner. 


3 


The final drawing is a visually dynamic composition in which most of the criteria 
are met in a fairly successful manner. 


2 


The final drawing may be visually dynamic but the criteria are met in a limited 
manner, or the criteria are met fairly well but the drawing is not very dynamic or 
visually exciting. 


1 


The final drawing meets the criteria in a minimal way. 


0 


Non-scorable; no attempt was made or design is unrelated to task. 
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Step 4: 



Review assessment items to identify areas that require change and revi- 
sion. Once test items are written, the development staff experienced in each 
arts discipline should review the questions and exercises first for content 
concerns. It is best that reviewers do this independently and confirm that 
questions fit the framework and are correctly classified according to frame- 
work specifications. External experts should also review items for sensitivi- 
ty and editorial concerns. Test questions should be revised accordingly. 



Activity: Convene the test development committee and 
external specialists to review and revise assessment questions. 



Review and edit based on the following qualities. 

Items/tasks are - 

□ Comprehensive and faithful in addressing the arts framework. 

□ Fair in testing content that all students had an equal opportunity 

to learn. 

□ Realistic with respect to the contexts and problems encountered in 
the arts field. 

□ Authentic in requiring student response/performance that represents 
the arts area being tested, i.e., dancing, singing, acting or creating. 

□ Feasible in terms of time, space, materials and stimuli required. 

□ Reasonable in allocating enough time for students to think and 
generate a response. 

□ Clear in directions for students that include contextual clues that enable 
them to apply the appropriate knowledge/skills to address the question or 
problem. 

□ Explicit in criteria that are understandable to students and which enable 
scorers to distinguish between diverse levels of performance. 

Also > review test items for - 

□ Elimination of controversial and sensitive topics. 

□ Balanced consideration of all student racial and ethnic groups. 

□ Sensitivity to students' varied backgrounds and regions. 
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Step 5: 



Pilot test the items and tasks in school classrooms. The main reason for 
doing this is to find out how well students understand the questions and 
directions. Another is to check on what further refinements should be made 
to the wording or format of the questions. You also will learn if the test ques- 
tions truly reflect current arts curricula and instructional practice. Particularly 
for the arts, the pilot can reveal, early on, the logistical possibilities and con- 
straints of using specialized arts materials, equipment and stimuli with cer- 
tain test items. Include in the trial a range of classrooms from across all 
regions of the state at targeted grade levels. 



Activity: Administer questions to students both 
in one-on-one and group tryout sessions. 



During the pilot sessions, make observations about the following: 

□ Clarity and complexity - are test directions and questions 
understood by students and teachers? 

□ Ease of administration - can questions be administered by all 
teachers using the same directions? 

□ Timing - is adequate time allotted for test items? 

□ Setting - can all classrooms accommodate the specific needs of the 
question, particularly production/performance tasks? 

□ Response sheet format - is there adequate space for student response? 
Are there unintended responses and why? 

□ Tools, materials, and stimuli - are these appropriate? adequately 
described? feasible to administer? cost effective? 
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Chapter 7 

Implement and Model 



K ey steps in the implementation phase include: 

1. Build your assessment instrument. 

2. Organize and administer the arts field test. 

3. Score student responses and review field test results. 

4. Coordinate and administer the assessment district or statewide. 

5. Analyze test data and report findings. 

Step 1 : 

Build your assessment instrument. This means creating the actual arts exam- 
ination booklets for students. At this point, you have a bank or pool of 
assessment items, in different formats, that you've piloted and revised for 
feasibility. If you plan to administer the arts test to more than one grade and 
in more than one arts discipline, you will need various versions or forms of 
the arts test. These alternate forms are assembled by combining blocks of 
items from your assessment pool. Think of a block as a related group of 
questions and tasks created by dividing your item pool for a grade and arts 
discipline into subsets. Here is a graphic to help you visualize these ideas: 



Raw, Unrefined Items 
are Developed 



Piloted Assessment Items in Various Formats 
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A more detailed list of key areas to consider in building your assessment 
instrument is shown in the following activity: 



Activity: Create arts test booklets (the assessment instrument) 
for the grades and arts disciplines to be assessed. 


Actions 


Helpful Hints 


Determine the assessment timing 


This is the total period of time during 
which a test booklet is administered to 
the student, e.gs., grade 4-60 minutes; 
grades 8 and 12-90 minutes. 


Determine if background 
questionnaires will be included 


These are used to collect supplemental 
information about students, teachers 
and schools such as student 
demographics and arts education 
experience; teacher years of teaching 
experience in the arts area being 
assessed and instructional practices; 
and school characteristics, enrollment, 
arts program features and related 
information. Questionnaires also 
may be included for students with 
disabilities and limited English 
proficiency.* 


Croup items/tasks into blocks 


Assessment items are grouped into 
blocks by dividing the item pool into 
related sets of questions for an arts 
discipline. The NAEP arts assessment 
blocks, for example, were designed 
around one or more stimuli such as a 
visual art work or video and consisted 
of related questions in different formats, 
i.e., multiple-choice and open-ended 
written questions with performance tasks. 
The NAEP art blocks took approximately 
25-30 minutes to complete. 


Assemble alternate test forms 


Different test forms for the arts are 
assembled by combining item blocks for 
a grade and arts area so that a student 
can take the test in a single arts discipline. 
Although you may have alternate test 
forms, all are intended for the same 
purpose such as to assess student 
achievement toward your district or 
state's arts standards or outcomes. 


• The NAEP 1997 Arts Report Card contains details on questionnaires. You will also find examples 
of thematic ' blocks " in each arts area that integrate performance tasks with other types of test items. 
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Step 2: 



Organize and administer the arts field test. The field test is generally more 
extensive than the pilot. It is administered to check the adequacy of testing 
procedures such as test administration, test scoring, as well as how students 
respond to the questions. The field test will provide further information 
regarding the clarity, level of difficulty, and timing of questions. Field test 
data will help you make final revisions and select the items to be used in the 
formal system-wide assessment. 



Activity: Manage your arts field test by addressing 
these responsibilities. 



Key Responsibilities 

1 . Develop the administrative procedures and manuals. 

2. Print field test booklets and questionnaires and 
produce other materials such as audio/videotapes, 
art reproductions and hands-on supplies. 

3. Select a broad and representative sample of schools 
and students to participate including students with 
disabilities and limited English proficiency.(Ensure 
that students in the field test meet established 
requirements for instructional time in the arts 
discipline assessed.) 

4. Establish a schedule of significant dates relevant to 
the field test. Your schedule should include dates 
for the arrival of 

a. principal test packages 

b. student arts test booklets, also dates for 

c. test administration 

d. deadline for return of tests 

5. Train staff and/or hire outside experts to administer 
the field test. 



One state printed 
booklets for their field 
test called "Test Sampler 
Drafts" with abbreviated 
test forms in the four 
arts areas for teachers to 
preview and use for 
instruction. See the 
introductory memo to 
the booklet on the 
following page. 



Some states have 
regional test 
coordinators to 
administer assessments; 

others contract with 
external agencies such 
as Westat or the 
Educational Testing 
Service (ETS) for 
guidance. Staff members 
from such organizations 
have extensive training 
in administering large- 
scale assessments and 
collecting data on 
student achievements. 
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As you organize your field test, take a look at the memo below that intro- 
duces the "test sampler draft" created for one state's high school arts assess- 
ment. Notice to whom the memo is addressed, what details it conveys, and 
how it encourages school districts and teachers to use the "test draft" to plan 
instruction. Also note that the sampler booklet is a way to release test items 
in advance of the field test for arts teachers to preview and use. 



example 



The State Department of Education 



To: District Superintendents, Superintendents, Administrators and 

Principals of Public and Nonpublic Schools, Coordinators and 
Teachers of Dance, Music, Theatre and Visual Arts 

From: Assistant Commissioner for Curriculum, Instruction and Assessment 

Subject: Arts Test Sampler Draft for the High School Graduation One-Unit 

Requirement in Dance, Music, Theatre and Visual Arts 
(Commencement-General Education Level) 

This test sampler draft is being released to assist school districts in planning 
their instruction and assessment of student achievement related to the 
State Learning Standards for the Arts. Administrators and teachers should be 
preparing students to demonstrate proficiency in at least one of the four arts 
disciplines, and to achieve at high levels in the respective assessment. The 
sampler addresses dance, music, theatre and visual arts and includes examples 
of test item types, formats (on demand written, on demand performance and 
performance/portfolio, and ancillary materials. This sampler is the result of 
the collaborative effort of the State Education Department, Intermediate 
School Districts, arts teachers, supervisors, national consultants, cultural 
partners and professional arts associations. Our State teachers and 
administrators have actively participated in the development and student 
tryout of the arts assessments. 

The Department will administer arts field tests in spring 2001 targeted 
for grade 9 students for selected samples of high schools. The purposes of 
the field tests are to further refine question formats and scoring materials 
in dance, music, theatre and visual arts. Once administered and returned to 
the Department, these field tests will be scored and evaluated so that 
assessments in dance, music, theatre and visual arts will be available for 
voluntary use in the 2001-2002 school year. 

We invite you and the members of your staff to provide the Department 
with feedback on these preliminary materials. On the last page of the 
sampler is a comment sheet that may be faxed to our office or mailed to 
the address provided. 
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Step 3: 



Score student responses and review field test results. This step highlights 
reliable scoring procedures as well as the recruiting and training of scorers. 
The information will help you process the data from your field test to select 
and refine test questions and scoring guides. 

Multiple-choice questions can be scored by scanning test booklets. 
However, in an arts assessment, there are often large numbers of student- 
generated responses to score for open-ended questions and performance 
tasks. Scoring these types of responses requires trained assessor judgment in 
order to achieve a high level of reliability. Reliability means the degree to 
which the scores of individual students are consistent over repeated appli- 
cations of the scoring criteria. To ensure reliability, you will need to recruit 
individuals who understand competent and creative performance in the arts 
disciplines assessed and train them to use appropriate criteria to discrimi- 
nate between student responses of different degrees of quality. 



Activity: Address these points to 
score student-generated responses. 



TOPIC 

Scoring Procedures 



POINTS TO COVER 

□ Use focused, appropriate scoring guides that match 
criteria emphasized in your arts assessment 
frameworks. (Use the initial draft scoring guides 
you developed with test items; based on the 
results of your field test, you can further refine 

the guides.) 

□ Recruit and train qualified scorers in the arts 
disciplines assessed. 

□ Monitor scorer consistency by reviewing each 
scorer's work to confirm that the scorer applies the 
scoring criteria consistently across a large number 
of responses. 

□ Document all aspects of your scoring procedures 
for technical reports. 



Recruiting and Training Scorers □ 

(Consider using arts discipline specialists 
along with measurement experts to 
conduct the training.) 



Present and discuss the question/item to be scored 
and its purpose and rationale in reference to 
your arts standards and framework. 



□ Present and clarify the scoring guide and criteria. 

□ Discuss the thinking behind the scoring guide with 
a focus on the criteria that distinguish the various 
achievement levels. Use concrete examples of 
student responses called "anchor" responses to 
illustrate each achievement/performance level.* 

□ Practice scoring a common set of sample student 
responses and discuss scores until scorers 
understand how to apply the guides. 



* Examples of student work , to illustrate different performance levels for a task, are included in The NAEP 
1997 Arts Report Card. Additional information about reliable scoring and training scorers can be found in 
recent editions of The NAEP Guide which describe large-scale assessment content and methods. 
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Reviewing Field Test results 



At the time of the field test, collect examples of student responses of vary- 
ing degrees of quality-representing a range of possible performance for 
each item. Select these as anchors or models for the achievement levels in 
your scoring guides. You can use the student examples in sessions to train 
scorers. Most importantly, you should review field test results by analyzing 
student responses to further design and improve your scoring guides. 
Consider, for example, if the criteria in your guides enable scorers to dis- 
criminate among diverse levels of student performance in a way that is not 
arbitrary. 

With concrete student results in hand, you now have the basis for important 
discussions with arts content specialists about the validity of the test ques- 
tions and tasks that your committees designed. Validity refers to the extent 
to which a test assesses what its designers or users claim that it assesses. Did 
the questions and items in the field test require students to apply the knowl- 
edge and skills, in your arts standards and assessment frameworks, to gen- 
erate competent and creative products/performances? Do the student 
responses indicate that the test assessed a particular arts discipline fairly and 
comprehensively? Examining the student responses around these critical 
issues, as well as the practical and administrative ones, will enable you to 
make further revisions to items that were field tested in your arts assessment. 
Consider, too, using outside groups such as arts and assessment experts to 
further revise and conduct final sensitivity and editorial reviews of your item 
blocks for each arts discipline assessed. 

Once you modify and select suitable items from the field test for your arts 
assessment, you are ready to: 

1. Prepare final versions of assessment materials, i.e., stimuli, such as 

audio and videotapes and art reproductions; also consumable art 

supplies, and 

2. Prepare, proof, and print arts test booklets and questionnaires. 



Step 4: 

Coordinate and administer the assessment either district or statewide. At 

this point in the process, you will focus on the administrative and logistical 
aspects of the arts assessment such as officially announcing the program sys- 
tem-wide and providing important dates and related details about school 
participation. How should you announce the program? What information 
should you convey? Below are three related examples of how one state with 
an arts testing program addressed these questions. Forms of communication 
like those shown are sent to schools and often posted at a state's web site in 
the arts and/or assessment listing of the site index. If you Included arts edu- 
cators and administrators in the development process and provided them 
with regular progress updates, there should be interest and enthusiasm 
among schools to participate in the assessment. 
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Activity: Use and adapt the following sample communication 
forms to announce your arts assessment program and to invite 
or require school participation, (Draw on the decisions you 
made in step 3 of the planning phase.) 



EXAMPLE A: general announcement at the start of the school year 

Fine Arts 

September 2001 

This school year, the State Board of Education will be offering a 
voluntary State Achievement Test in the Fine Arts for grades 9 
and/or 1 0. High schools will have the option of administering the 
voluntary test during the State's Achievement Testing period, 

April 1 - 12, 2002, or during April 24 - 25, 2002. Schools wanting 
to enroll for the 9 and/or 1 0 voluntary arts test may do so on the 
Enrollment Form that will be sent to schools this fall. 



EXAMPLE B: Letter to School principals - voluntary participation 



July 6, 2002 

Dear High School Principal, 

This is to inform you that the State Board of Education will offer a voluntary 
Arts Standards Achievement Test in April 2003, for the high school level. 

The assessment will be offered in the areas of dance, music, theatre and 
visual arts and participating students will take the test in a single arts area. 

This voluntary arts assessment is for students in grades 9 and/or 10. The test is 
aligned with the State Arts Learning Standards and takes approximately 90 minutes 
to complete. The results are for local use only-to determine how well students 
are meeting the arts standards. If your school participates, you have the option 
to administer the examine during the State Achievement testing period of 
April 1- 1 2, or during April 24-25. 

To enroll your students, complete the Voluntary Test Enrollment Form that is 
enclosed with this letter. You can either fax or mail the form to the Assessment 
Division at 

If you choose to enroll your school, you will receive testing materials in early 
March. Please keep the materials secure until testing time. After the test is 
administered, return all testing materials to the assessment coordinator 
by April 29, 2002. 

Please share this information with the fine arts teachers in your building. If you 
have questions, contact the Assessment Division at . 



Sincerely, 



Assessment Division Coordinator 
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EXAMPLE C: arts test enrollment form for high school, target grades - 9 and/or 10 



State Voluntary (or Required) Arts Test 

Enrollment Form } 

Fax the completed form to by October 5, 2001 

Address and Contact Information I 

District Name: | 

School Name: 

School Address: II 

List Art(s) Teachers and the Arts Discipline taught: 



Contact Person: (All test materials will be sent to the attention of this person al the school address) 



Participating Student Enrollment Counts: 

Grade 9 Grade 10 

Dance 

Music 

Theatre 

Visual Arts 

Signature of Principal 



Step 5: 

Analyze test data and report findings. Now that you have implemented 
your assessment plan, it's time to maximize the results of your work. Think 
again about the goals and how the results of your program can impact on 
your stakeholders. Most districts and states, perhaps like yours, design arts 
assessments to enable students to make progress towards the achievement 
of arts standards. Towards that end, the feedback and reports on test per- 
formance should be in clear, understandable terms that students and teach- 
ers can respond to and use to improve achievement. 

Put simply, you want to collect and analyze the test data in such a way as 
to make the information meaningful to students, teachers, parents, adminis- 
trators, policymakers, and the public. This may require a different type of 
report for each audience. But first, you have to gather and process the 
assessment results. To assist with this, you should have the support of the 
measurement expert in your system or one from an outside education 
research organization. The assessment data collected will be translated into 
numbers and perhaps can be conveyed visually in a graph or chart. 
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How do you hope to use the results of your assessment? What areas and arts 
education program needs could the test findings address for your situation? 
The activity below will help your advisory and development committees 
answer these questions. Selecting the target uses for your assessment find- 
ings will enable the measurement consultant to determine the best cate- 
gories for organizing the test data, such as by a particular art standard, arts 
discipline, grade level and/or content strand. Student individual and group 
scores then will be summarized in a certain category according to best prac- 
tice procedures for measurement scaling. 



Activity: Determine and pripriti^e the internal and external uses for your 
assessment findings based on the arts education program needs in your 
district or state. (A few examples will help get your discussion started,) 



Assessment Results 

Internal Uses External Uses 



Motivate students to improve 
achievement towards arts standards 


Garner support for an innovative 
school arts education effort 


Focus professional development goals 
for district or state arts teachers 


Retain or increase funding for arts 
resources and programs 


Increase student instructional time in the 
arts for a grade level and/or arts discipline 


Gain community advocacy and 
recognition for arts education 


Strengthen a targeted arts learning 
area or arts discipline program 


Achieve designation as a model site 
for a new arts initiative 



Determining the uses for your findings also will help you decide what kinds 
of reports to write and for which audiences. Be sure to generate your reports 
in a timely fashion so as not to lose the impact of the testing event. NAEP 
produces a comprehensive set of reports for subject areas tested, targeted to 
specific audiences. Here are a few the different types of reports NAEP pro- 
duces that may be relevant to your situation: 

HIGHLIGHTS REPORT 

In a non-technical style, this type is intended to answer general ques- 
tions about the assessment that a parent, school board officer, or inter- 
ested community member might ask. Think of it as addressing the "fre- 
quently asked questions" you faced throughout the program. You would 
also include promising student achievement results in focus areas to 
showcase the value of the arts to student learning. 

INSTRUCTIONAL REPORT 

This is the kind of report to produce for classroom arts teachers, princi- 
pals, and other school administrators. Not only would it focus on find- 
ings related to arts curriculum and standards, but also it would 
include sample test items, performance tasks, and scoring guides that 
teachers could use in their classrooms. Additionally, it could show stu- 
dent test results by school or district. 
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STATE REPORT 

This type will be useful to policymakers and state department staff 
responsible for arts education. It would focus on arts assessment find- 
ings about overall program strengths and weaknesses in the arts disci- 
plines tested. Findings would relate to student achievement of arts stan- 
dards statewide and district-wide. 

TECHNICAL REPORT 

This report type is for documenting all the details of your assessment, 
including the design, item development, data collection, and analysis 
procedures. It does not report test results, rather it describes how you 
got them. 

For further information on reporting and using test results , see the CCSSO publica- 
tion i. Handbook for Professional Development in Assessment Literacy , included in 
the references on page 47. 
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Chapter 8 

Benefits of Assessment to Arts Education 



W hether or not it is linked to large-scale accountability systems, 
assessment holds great value for a wide variety of arts education 
audiences. Arts assessment can help curriculum designers and 
program administrators articulate goals and standards, identify program 
strengths and weaknesses, and indicate key targets for the allocation of 
resources. For parents and students, assessment results provide a profile of 
student learning. Young learners need feedback in order to refine and redi- 
rect their efforts. 

For teachers, assessment can provide important information needed for suc- 
cess in the classroom. Results can lead teachers to answer the question: 
"Are my students actually learning what I am setting out to teach?" Effective 
assessment is an integral part of instruction and has its greatest value when 
it is closely tied to classroom practice. 




Assessment as Professional Development 

Teachers can play an important role in the design and development of arts 
assessment, increasing their own knowledge and skills in the process. As 
one researcher from Educational Testing Service states: 

We may have all the latest theories, and we may know all about 
construct validity and reliability , but teachers know the classroom 
and the teacher's perspective is invaluable to us. (ETS Trustee's 
Colloquy , "Performance Assessment," 1995, p. / 6.) 

By serving on state and local committees to plan assessment frameworks, 
teachers have the opportunity to closely examine curricular standards and 
clarify goals for instruction. By helping to design effective assessment tasks, 
teachers learn good teaching practice. The National Center for Research on 
Evaluation, Standards, and Student Testing (NCRESST) recommends criteria 
for judging effective assessment tasks that match questions teachers should 
ask of their own instruction. (Herman, Ashbacher and Winters, A Practical 
Cuide to Alternative Assessment, ASCD, 1 992, p. 42) The following exam- 
ples illustrate the similarities between good assessment tasks and good 
teaching: 

♦> Do tasks pose an enduring problem that students are likely to face 
in school or in their future lives? 

❖ Will tasks be seen as meaningful and challenging by students, par- 
ents, and teachers? 

❖ Will tasks be engaging to students so that they will be motivated to 
do their best work? 
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Like their classroom colleagues, arts educators need professional develop- 
ment to stay current with content knowledge and to hone their instruction- 
al skills. Yet in most school districts, resources for the professional devel- 
opment of teachers in the arts are scarce. By involving teachers in assess- 
ment design, you can use local or state accountability funding for arts pro- 
fessional development purposes. 

For example, in the Vancouver, Washington public schools, local arts teach- 
ers became involved in the district's early experiments with performance 
assessment. When the call went out from the district's Office of 
Accountability and Research for test item development, arts educators 
signed up along with their colleagues in science and writing. Led by district 
researchers, arts teachers took a shot at designing assessment tasks using the 
"Ready, Fire, Aim" approach suggested by Grant Wiggins. They targeted 
objectives, wrote prompts, designed protocols to capture student work using 
video, and tried out performance assessment tasks with teachers and stu- 
dents throughout the district. Later, teachers came together to look collec- 
tively at the student work, to discuss rubrics and issues of quality perform- 
ance, and to revise and refine the tasks. Most importantly, teachers returned 
to their classrooms with increased clarity about arts learning and a wealth of 
new strategies to guide instruction. And, it was all paid for by the Office of 
Accountability. 

Reflective Practice for Arts Educators 

Even when not framed as part of an extensive district wide professional 
development effort, arts assessment can provide teachers a forum for reflec- 
tive practice. Music educators have long gathered at festivals to collectively 
adjudicate student work. On a less formal level, two or three arts teachers 
can gather on their own to look together at the work of their students. 
Departments can use reflective practice protocols such as those developed 
by Steve Seidel and his colleagues at Harvard's Project Zero to "wonder" 
about student art works. 

These conversations about student work — both formal and informal — can 
lead to the creation of collectively designed rubrics. With actual samples of 
student work in front of them, teachers can be encouraged to discuss and 
consider such questions as: 

❖ What example of student work stands out as high quality? 

❖ What features lead me to consider the work to be "excellent"? 

❖ Did successful students approach the task in different ways than 
unsuccessful students? 

Clearly divorced from the pressures of grading, teachers can openly discuss 
these important assessment issues with colleagues, building a community of 
arts educators who come to have shared understandings of excellence. 
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Arts Assessment and Advocacy 



Local arts educators often tire of the seemingly endless need to "justify" the 
arts to school boards, principals, or parents. Yet, until the American public 
gains significant engagement in the arts, it seems likely that "advocacy"— 
making a compelling case for the importance of arts learning for every child 
—will continue to be an important part of arts education efforts. 

A thoughtfully conceived arts assessment program can assist advocacy 
efforts by increasing public understanding of arts learning. Assessment can 
illuminate the often-elusive qualities of the arts. For example, performance 
assessments like those developed for the NAEP involve capturing student 
work on video for later viewing by raters. Local arts educators are discov- 
ering the value of sharing these video snapshots of learning (along with arti- 
facts or portfolios) with school leaders and administrators, giving policy 
makers a look "inside" student performance in the arts. By sharing the 
"secrets" of arts learning— by clarifying the processes of critical analysis or 
creative composition that take place in the minds of students as they make 
art— arts educators can take policy makers further than the concert or gallery 
reception, helping them to discover that the arts have value beyond orna- 
ment or entertainment. 

In order to maximize the potential of assessment for advocacy purposes, 
local arts educators might consider the following strategies: 

❖ Lobby for the inclusion of the arts when reporting local student 
achievement data; 

❖ Encourage local school boards to request a full and complete pic- 
ture of student achievement in all of their school programs, includ- 
ing the arts; 

❖ Ensure that reports or presentations about the arts focus on student 
learning; 

❖ Design assessments that give evidence of students using their minds 
well while creating, performing and responding to the arts; 

❖ Consider using the voices of teachers and, where appropriate, even 
students during informal arts assessment reports. A student describ- 
ing their own process, intent and purpose for art making can pro- 
vide compelling insights for school leaders. 

The primary purpose of assessment in the arts is, finally, to improve teach- 
ing and student learning. A well-planned advocacy program linked to 
assessment efforts can help ensure that the important work of arts education 
will continue. 
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Appendix 



T he following performance exercise in dance for grades nine through 
twelve was developed by the SCASS Arts Education Assessment 
Consortium as an example of an assessment that might be done by a 
teacher in a classroom using the NAEP framework and the three arts process- 
es of Creating, Performing, and Responding. 

Unlike the NAEP Arts Assessment that consisted of "on-demand" exercises, 
or blocks of questions completed within a prescribed timeframe as part of 
the total assessment time, this "curriculum-embedded" example was 
designed to be conducted over two class periods. Other examples of assess- 
ments done "over-time" would be process portfolio collections and "best- 
work" portfolios. 

The CULTURAL TRADITIONS exercise found on the following pages is 
built from the National Standards for Arts Education and follows the 
SCASS/Arts format that includes: 

1. The relevant national standards and the performance indicators, 

2. A task description and summary of the activity, 

3. The purpose and content being assessed, 

4. A background and context for the activity, 

5. Set-up and special instructions for the facilitator, 

6. Materials and equipment required, 

7. A script and administrative directions, 

8. Criteria for scoring the work (shared with the students before the 
activity), 

9. A student response sheet for collecting a narrative description of 
the student work, and 

10. A scoring sheet for teachers. 

The SCASS Arts Education Assessment Consortium has a collection of per- 
formance exercise examples available and has been working on a web- 
based, assessment item development program. The items are being placed 
in a searchable database and will be available for teachers and other arts 
educators to use in constructing arts assessments. 
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Dance Exercise, Grade Level 9-12 

Cultural Traditions 



NATIONAL STANDARDS ASSESSED 

1. Identifying and demonstrating movement elements and skills in performing dance 

Students: 

h. describe the action and movement elements observed in a dance, using appropriate move 
ment! dance vocabulary. 

2. Understanding choreographic principles, processes, and structures 

Students: 

d. demonstrate the ability to work cooperatively in a small group during the choreographic 
process. 

3. Understanding dance as a way to create and communicate meaning 

Students: 

d. demonstrate further development and refinement of the proficient skills to create a small 
group dance with coherence and aesthetic unity 

TASK DESCRIPTION/SUMMARY 

Total Time for Block: Two class periods: one session to explain the task and create the dance, and one ses- 
sion to perform the dance and complete the response form. 

Students will collaborate to create and perform an original dance based on knowledge of another culture’s tra- 
ditions. Students must have done some research into a culture’s dance traditions prior to creating the dance. 
The dance will contain a clear beginning, middle, and end, and include appropriate rhythmic and/or vocal 
accompaniment. Students will respond by identifying the elements of time, space, and force and will explain 
how and why these elements were included in their composition. In an open-ended extended question, the 
students will describe how the traditions of a culture have been expressed in their dance through: 

Space: shape, level, direction, size, focus, pathway 

Force: attack (sharp/smooth), weight(heavy/light), strength(tight/loose), flow (free flowing, bound, or 
in balance) 

Time: beat, accent, speed(tempo), duration(long/short), rhythmic pattern 

PURPOSE OF ASSESSMENT 

Students will be assessed on: 

• exploring and improvising on prior knowledge of another culture’s traditions 

• composing/creating a group dance 

• performing a group dance 

• responding to their dance through description of movement elements using appropriate vocabulary. 

BACKGROUND/CONTEXT 

Students will need to have an understanding of traditions in various cultures (Native American, Latin 
American, African, Asian, European). Students will need to know dance terminology. Students will need to: 

• remember extended movement sequences 

• project while performing 

• create combinations with variations of space, time and force elements 

• either reflect authentic movements or abstract the essence of dance movements of the culture. 
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SET-UP AND SPECIAL INSTRUCTIONS FOR THE FACILITATOR 



Students will work in groups of 3-4 that are self-selected or teacher assigned. 

Students will have been instructed to wear loose, comfortable clothing and bare feet or sneakers— no stock- 
ing feet. Students should have their name displayed on a 5"x8" card with a large marker and attached to their 
clothing in a manner to allow those viewing the video tape later to identify students for scoring. 

In one corner of a large, open space (preferably a dance studio, theatre, or gym), have a technician set up and 
operate a VCR and television set. Leave as much free space as possible for students to dance. 

Instructor administering assessment should display enthusiasm and a supportive, encouraging approach. 
Instructor should speak naturally to students and answer questions as fully as possible without supplying 
ideas/answers, etc. 

Final performances must be videotaped. To ease the scoring process, students should announce the culture 
of their dance, the group number, and their individual name before the performance. Read the booklet enti- 
tled "Videotaping Performance Assessment" for further guidelines. 

Once dance is videotaped, students will watch the video and be given remainder of assessment time to com- 
plete written task. Instructor will collect written student evaluations of dance and use these and video of ses- 
sion for final assessment of each of the four students. 

MATERIALS/EQUIPMENT REQUIRED 

• Video camera (and operator) 

• TV/VCR 

• Blank tape 

• Tripod, or person to operate camera 

• 5" x 8" cards, large markers, and safety pins for name labels 

• Note paper for responding to video 

• Large open space where students can move safely 

• Written task directions for students displayed in room 

SCRIPT AND ADMINISTRATION DIRECTIONS 



DAY ONE 

SCRIPT: I am going to ask you to create an original dance based on the traditions of one of the following cul- 
tures, African, Asian, European, Latin, or Native American. Your dance will be approximately 2 minutes in 
length. As a group, you will generate movements for a dance that either reflect 1) authentic movements of 
the culture selected or 2) an abstraction of the essence of movement qualities of the selected culture. 
Separately you will explore and improvise movement that expresses aspects of your chosen culture. The 
dance should have a clear beginning, middle, and end, and the improvised movements you select should 
reflect and relate to the culture. As a group, create and include vocal and/or rhythmic sound to accompany 
your dance. 

SCRIPT: You will have two sessions to do the following: 

• On day one, you will improvise movements alone and then you will be placed in a group to create 
a group dance and create vocal and/or rhythmic sound to accompany the dance which will be video 
taped for scoring later. 

• On day two, you will rehearse and perform the dance for video recording and respond in writing to 
your performance on a form provided. 
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SCRIPT: Based on your knowledge of the dance movement of the culture you are working with, improvise 
movement that either reflects authentic movements of the culture you have selected or abstracts the essence 
of movement qualities of the culture selected. You will have ten minutes to do this. 

[Allow students 70 minutes to improvise.] 

SCRIPT: In your group create and refine a 2 minute dance incorporating your improvised movements. 
Develop a vocal and/or rhythmic sound accompaniment for your dance. You have 45 minutes to complete 
this task. 

[Allow students 45 minutes, announcing the time in 7 5 minute intervals to help them pace themselves.] 

DAY TWO 

SCRIPT: You will have 5 minutes to warm up and rehearse your dance. Each group will do their dance and 
it will be video taped. After all groups perform, the video tape will be replayed so that each student can com- 
plete the response sheet which reflects on how the dance incorporated dance elements. When your group is 
ready to perform, announce the culture of your dance, the group number, and each individual name before 
the performance. 

[ Videotape the second day performances, rewind the tape, gather the students around the monitor and pro- 
vide the response sheets and pencils. [ 

SCRIPT: Please watch the video of your dance performance. Using the paper and pencil provided, please 
analyze your performance on the response sheet. You will have 10 minutes. 

1) Using appropriate dance vocabulary, identify and describe the dance elements you used and varied in 

your dance. 

2) How do your movements and use of dance elements relate to or reflect the culture you chose? 



[After 70 minutes, the facilitator calls time.] 



CRITERIA FOR SCORING 

The rubrics on the following pages provide the basis for scoring the student work and use an analytic format 
which allows options in scoring. The rubrics are also found on a score sheet on a following page. Each com- 
ponent can be scored separately to provide diagnostic information for teachers/students. For a holistic score, 
add component scores together, then determine a range of scores for each performance level. 
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CREATING 



Level 4 


• The dance has clear beginning, middle, and end. 

• Use of time, space and force elements enhances the theme. 

• Vocal and/or rhythmic accompaniment are fully integrated into 
the dance which clearly reflects the cultural tradition. 


Level 3 


• The dance has a clear beginning, middle, and end. 

• Efficient use of time, space and force elements relates to 
the theme. 

• Vocal and/or rhythmic accompaniment relate to the dance 
and reflect the cultural theme. 


Level 2 


• The dance has unclear delineation between beginning, 
middle, and end. 

• Efficient use of time, space and force is evident but does not 
clearly relate to the cultural theme. 

• Vocal and/or rhythmic accompaniment exist and at times 
reflect the cultural theme. 


Level 1 


• The dance shows little evidence of beginning, middle, and end. 

• Use of dance elements is minimal (time, space, force). 

• Disconnected or little use of vocal and/or rhythmic 
accompaniment. 



PERFORMING jl 


Level 4 


• Student performs with full commitment and intensity (projects 
while performing; remembers extended movement sequence). 

• Student demonstrates principles of alignment, balance and 
control. 

• Student uses variety in elements of space, time and force. 

• Student uses movements which blend with the accompaniment 
throughout the performance. 

• Student is always aware of self and others and responds to 
changing spatial relationship. 


Level 3 


• Student performs expressively with commitment (projects 
while performing; remembers extended movement sequence). 

• Student demonstrates principles of alignment, balance and 
control. 

• Student uses some variety in elements of space, time and force. 

• Student uses movements which blend with the accompaniment 
occasionally. 

• Student is aware of self and others and responds to changing 
spatial relationships. 


Level 2 


• Student performs with some projection, but has difficulty 
remembering the sequences. 

• Student demonstrates principles of alignment, balance, and 
control with some commitment. 

• Student varies space, time, and force elements infrequently. 

• Student is aware of self and others and occasionally responds 
to changing spatial relationships. 


Level 1 


• Student performs with no projection/doesn't remember 
extended movement sequences. 

• Student demonstrates minimum physical commitment and 
control. 

• Student has no variation in space, time, and force elements. 

• Student demonstrates little or no awareness of spatial 
relationships. 



Envisioning Arts Assessment: A Process Guide for Assessing Arts Education 



53 




4 9 BEST COPY AVAILABLE 



RESPONDING 11 


Level 4 


• Student expands and elaborates in describing key elements 
using dance vocabulary: Space-level, direction, shape, 
pathway, focus, range; Time-beat, accent, tempo, duration; 
Force-attack: sharp/smooth, weight: heavy/light, 
strength: tight/loose, flow: free-flowing, bound, or balanced 

• Student draws parallels, discussing more than one connection 
to cultural traditions. 


Level 3 


• Student identifies and describes key elements using dance 
vocabulary: Space-level, direction, shape, pathway, focus, 
range; Time-beat, accent, tempo, duration; Force-attack: 
sharp/smooth, weight: heavy/light, strength: tight/loose, 
flow: free-flowing, bound, or balanced 

• Student makes a connection to cultural traditions. 


Level 2 


• Student identifies some of the elements using dance vocabulary: 
Space-level, direction, shape, pathway, focus, range; Time-beat, 
accent, tempo, duration; Force-attack: sharp/smooth, weight: 
heavy/light, strength: tight/loose, flow: free-flowing, bound, 

or balanced 

• Student attempts to make a connection to cultural traditions. 


Level 1 


• Student describes dance using general vocabulary rather 
than dance vocabulary. 

• Student describes dance with no relationship to cultural 
traditions. 



— — ^ ^ 

GROUP DYNAMICS 1 i 


Level 4 


• Student actively participates in group by sharing ideas/ 
movements, listening and responding, leading initiatives, 
and expanding on ideas of others. 


Level 3 


• Student participates in group by sharing ideas/movements, 
listening and responding, following initiatives of others, 
and accepting ideas of others. 


Level 2 


• Student participates in a passive manner by accepting ideas/ 
movements of others, listening and responding in a limited 
manner. 


Level 1 


• Student shares few ideas and shows little response to ideas 
of others. 
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STUDENT RESPONSE SHEET 



Student Name Date 

Student Group Number 

Cultural Tradition Selected 

Teacher Name 

1) Using appropriate dance vocabulary, identify and describe the dance 
elements you used and varied in your dance. 



2) How do your movements and use of dance elements relate to or reflect 
the culture you chose? 
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TEACHER'S SCORING SHEET FOR CULTURAL TRADITIONS EXERCISE 



Student's Name 



Student's Group Number 



Date 



Cultural Tradition Selected 



Teacher's Name 



Mark a single score for each of the four components (Creating, Performing, Responding, and Croup Dynamics) in the space next to the 
description that best fits the student's performance and again in the box at the bottom of each column. 



LEVEL SCORE 



CREATING 



SCORE 



PERFORMING 



SCORE 



RESPONDING 



SCORE GROUP DYNAMICS 



The dance has a clear 
beginning, middle, and end. 
Use of time, space and 
force elements enhances 
the theme. 

Vocal and/or rhythmic 
accompaniment are fully 
integrated into the dance 
which clearly reflect the 
cultural tradition. 



■ Performs with full commitment/ 
intensity (projects while performing; 
remembers extended movemenl 
sequence). 

■ Demonstrates principles of 
alignment, balance and control. 

■ Uses variety in elements of space, 
time and force. 

■ Uses movements which blend with 
the accompaniment throughout the 
performance. 

■ Is always aware of self and others 
and responds to changing 
relationships. 



Student expands and 
elaborates in describing key 
elements using dance 
vocabulary. Space-level, 
direction, shape, pathway, focus, 
range; Time-beat accent, tempo, 
duration; Force-attack: sharp/ 
smooth, weight heavy/light, 
strength: tight/loose, flow: free- 
flowing, bound, a balanced. 
Student draws parallels, 
discussing more than one 
connection to cultural 
traditions. 



1 Student actively 
participates in group by 
sharing ideas/ 
movements, listening 
and responding, 
leading initiatives, and 
expanding on ideas 
of others. 



The dance has a clear 
beginning middle, and end. 
Efficient use of time, space 
and force elements relates 
to the theme. 

Vocal and/or rhythmic 
accompaniment relate to 
the dance and reflect the 
cultural theme. 



1 Performs expressively with 
commitment (projects while 
performing; remembers extended 
movement sequence). 

1 Demonstrates principles of 
alignment, balance and control. 

1 Uses some variety in elements of 
space, time and force. 

■ Uses movements which blend with 
the accompaniment occasionally. 

1 Is aware of self and others and 
responds to changing spatial 
relationships. 



Student identifies and describes 
key dements using dance 
vocabulary Space-level, 
direction, shape, pathway, focus, 
range; Time-beat, accent, tempo, 
duration; Force-attack: sharp/ 
smooth, weight heavy/light, 
strength: tight/loose, flow: free- 
flowing, bound, or balanced. 
Student makes a connection to 
cultural traditions 



Student participates in 
group by sharing ideas/ 
movements, listening 
and responding, 
following initiatives 
of others, and accepting 
ideas of others. 



The dance has unclear 
delineation between 
beginning, middle, and end. 
Efficient use of time, space 
and force elements is 
evident but does not clearly 
relate to cultural theme. 
Vocal and/or rhythmic 
accompaniment exists and 
reflects the cultural theme. 



• Performs with some projection, but 
has difficulty remembering 
sequences. 

Demonstrates principles of 
alignment, balance, and control with 
some commitment. 

Varies space, time, and force 
elements infrequently. 

Demonstrates limited awareness of 
self and changing spatial 
relationships. 



Student identifies some of the 
elements using dxnce 
vocabulary: Space-level, 
direction, shape, pathway, 
focus, range; Time-beat, accent, 
tempo, duration; Force-attack: 
sharp/smooth, weight heavy/ 
light strength: tight/loose, flow: 
free-flowing, bound, or balanced. 
Student attempts to make a 
connection to cuRural traditions. 



1 Student participates 
in a passive manner 
by accepting ideas/ 
movements of others, 
listening and 
responding in a 
limited manner. 



The dance shows little 
evidence of beginning, 
middle, and end. 

Use of dance elements is 
minimal (time, space, force). 
Disconnected or little use 
of vocal and/or rhythmic 
accompaniment. 



Performs with no projection, 
doesn't remember movement 
sequences. 

Demonstrates minimum physical 
commitment and control. 

Has no variation in space, time, 
and force element 
Demonstrates little or no 
awareness of spatial relationships. 



Student describes dance using 
general vocabulary rather than 
dance vocabulary. 

Student describes dance with no 
relationship to cultural traditions. 



Student shares few 
ideas and shows little 
response to ideas of 
others. 



Not Scorable - Evidence is 
not relevant to task 
requirements or is too 
minimal to rate student 
performance. 



• Not Scorabte - Evidence is not 
relevant to task requirements or is 
too minimal to rate student 
performance. 



• Not Scorable - Evidence is not 
relevant to task requirements or 
is too minimal to rate student 
performance. 



Not Scorable - Evidence 
is not relevant to task 
requirements oris too 
minimal to rate student 
performance. 



Score for Creating 



Score for Performing 



Score for Responding 



Score for Group Dynamics 
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